Short Abstract: State estimation techniques effectively provide mean state estimates. 
However, the theoretical state error covariance matrices provided as part of these techniques 
often suffer from a lack of confidence in their ability to describe the uncertainty in the estimated 
states. By a reinterpretation of the equations involved in the weighted batch least squares 
algorithm, it is possible to directly arrive at an empirical state error covariance matrix. The 
proposed empirical state error covariance matrix will contain the effect of all error sources, 
known or not. This empirical error covariance matrix may be calculated as a side computation 
for each unique batch solution. Results based on the proposed technique will be presented for a 
simple, two observer and measurement error only problem. 
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Background 


• There are various estimation techniques: 

- Batch 

- Sequential (Kalman) 

• Provide a theoretical state error covariance matrix describing 
estimate uncertainty under perfect process knowledge, maybe with 
process noise. 

• Theoretical state error covariance matrices: 

— Do not include all error sources. 

- Too small but may be too large if improperly corrected. 

- Not trusted as a quantitative description of the state error. 

• "Filter" error covariance matrices are, at best, qualitative estimates 
of the error. "Is the estimate good enough for..." 

• There appears to be a lack of reference to any formal empirical state 
error covariance matrix for such estimation processes. 
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What about an empirical error covariance matrix? 


• An empirical state error covariance matrix may be determined. 

• This matrix will include all error sources, known or not! 

• Why? 

- Actual observations contain true measurement errors. 

- Estimated measurements contain all other errors, known or not. 

- Therefore, measurement residuals contain all errors, known or not. 

• What about unknown bias? 

- An empirical covariance matrix will not eliminate bias problems. 

- Biases will be part of the empirical uncertainty of the estimate. 

- Current theoretical covariance excludes bias altogether. 
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For Batch Estimation 


• The empirical matrix is consistent with existing tools. 

• The empirical matrix only requires the addition of side computations 
to any existing batch estimator. 

• The empirical matrix is a straight forward statistic of a sample 
measurement process. 

• The empirical matrix comes, as any statistic should, with a path to 
confidence intervals for elements of the matrix. 



Joe Frisbee 


3 


Dynamics 


Typical Batch Estimation Presentation 


• Minimize the total variance cost function of the weighted 
squares of the residuals: 

4*)=^(y-Hx) r W(y-Hx) 

• Standard form of the solution: 

x = (H r WH ) _1 H T Wy 

• Identified error covariance matrix of the estimate: 

p = (h t wh)"‘ 

• This is usually where the story ends. 
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Empirical Error Covariance Matrix 


Rewrite the cost function using summation notation and in the form 
of a mean rather than a total variance: 


= (y, -H,x) 

Follow the usual procedure, only keep the "N" and summation 
notation: 
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Empirical Error Covariance Matrix 


• Identify the coefficient term as the population error covariance 


• For the solution equation, combine all of the terms inside the 
summation and identify the summation argument, x . , as an 
effective state vector measurement residual. (Note: the i-th effective 

state vector residual is not the generalized inverse solution to the i-th measurement 
residual expression.) 


matrix associated with the traditional batch estimation algorithm: 
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Empirical Error Covariance Matrix 


From the last equation, it follows directly that the empirical 
(sample) population error covariance matrix is given by: 


P ' = N- 


3 z [ft “ - xf . 

f 1=1 


After the usual iterative process for batch estimation, the estimated 
correction is essentially zero and the above may be approximated 
by: 


~ 1 ^ 

P„= — > lix 


-i -±. 
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Empirical Error Covariance Matrix 


From statistics, the covariance of a sample mean is just the sample 
covariance of the population divided by the number of samples: 


p=-L y 

N 2 ^ 


i=\ 


T 


This is the empirical error covariance matrix for the batch estimation 
process written in terms of effective state measurement residuals. 

What does this expression look like using the original terms in the 
state update equation? 
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Empirical Error Covariance Matrix 


Using some of the original expressions, the empirical error 
covariance matrix may be written as: 


P = 


1 


N 


2 P 


Z( HfW^yfW^,) 


i = 1 


P 


Using the relationship between the population covariance and the 
traditional state error covariance, the empirical form of the batch 
estimate error covariance matrix may be written as: 


P = P 


£,{n!w,y,y!w,H,) 


i = 1 


4 - + 

,ee© 
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Empirical Error Covariance Matrix 


This last equation may be written in appended matrix form as: 

P = PH T WYWH P 

Note: the center matrix, Y, is not the outer product of the 
traditional appended residual vector with itself. Y is an appended 
block diagonal matrix with each block being the outer product of an 
individual measurement residual vector with itself: 


Y = 


4 - + 

,££© 
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Empirical Error Covariance Matrix 


Points to note 


• All information necessary for computation of an empirical 
state error covariance matrix is present within any existing 
batch estimate process. 

• Assuming that the measurement residual weighting matrix is 
the inverse of the measurement error covariance, the 
expected value of the empirical error covariance matrix is the 
same matrix as is usually computed for the traditional batch 
estimate: 
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Example: Stationary Triangulation 


• Two range only observers ranging a target object. 

• Sample range measurements were generated using given but 
different sensor noise values for each of the two sensors. 

• Standard batch process subject to faulty range measurement 
weights: the sensor uncertainties were swapped prior to data 
processing. 

• Data generated: 

- Theoretical, 2 sigma, error ellipse using correct weights. 

- A field of multiple predicted locations generated by repeating the 
experiment numerous times w/ calculated 2 sigma covariance matrix. 

- Empirical, 2 sigma, error ellipse from the measurement residuals 
under the influence of the erroneous measurement weights. 

- Theoretical, 2 sigma, error ellipse using swapped weights. 
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Example: Stationary Triangulation 
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Error Covariance Confidence Interval 


• Sample processes, by their very nature, provide statistically 
uncertain results for each parameter determined through the 
sampling process. 

• A standard part of most statistical analyses is to determine the 
uncertainty associated with an estimated parameter. 

• The batch estimation procedure provides a theoretical 
estimate of the uncertainty in the estimated state and, as just 
shown, an empirical estimate of the state uncertainty. 

• It is also possible to directly consider the uncertainty in the 
estimates of the error covariance matrices themselves. 
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Error Covariance Confidence Interval 


Recalling the earlier form of the empirical error covariance matrix: 


P = P 


t(H T W.y.y T WH.) 

\ l l l l / 


i-\ 


If the weighting matrix is the inverse of the expected variance of the 
measurement, then each residual maybe rewritten in terms of a 
normalized vector and the square root matrix of the expected 
variance corresponding to each measurement's error. 


y =W. ^ u. = S.u. 

• / i i i i i 
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Error Covariance Confidence Interval 


After substitution, bringing the traditional matrix inside the sum and 
simplification, the empirical error covariance matrix is of the form: 


P = £(b iU ,uX) 


i=l 

Let b i:k represent any explicit row, k, of B, above. It can then be 
shown that the contribution of the i-th observation to the row m 
and column n element of the prior expected value of the error 
covariance matrix is: 


e\p ] = b b T = P 

L v.mn J v.m v.n v.mn 


4 - + 
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Error Covariance Confidence Interval 


It can also be shown that the expected value of the contribution of 
each observation to the variance of each component of the prior 
expected value of the error covariance matrix is: 


v\p ]=(b b T Xb b T )+(b b T ) 

l 1 i\mn J yi:mri:m f\i\n u i\n /' \i\m u i\n ) 


With the final results: 


r N 

E\P Uyz 3 =P 

L mn J v.mn mn 


i=l 


v\p 1=v[p 1 

L mn J / j L l - mn J L mn J 
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Error Covariance Confidence Interval 


Considering the diagonal terms, it is possible to write each of the 
previous results in terms of a gamma distributed variable with "to 
be determined" shape, K, and scale, 0, parameters: 

p =eIWk 0 Y| = K 0 

mm L 1 V lv mm ’ u mm /J mm u mm 


V[P_] = V[r(K_,0„)] = KJ 


mm 


It is possible to solve for both shape and scale parameters. These 
two parameters define the distribution of the uncertainty of the 
diagonal covariance matrix element with which they are associated 
under the prior expectations of the measurement errors. 


-f J: 
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Error Covariance Confidence Interval 


What about an off-diagonal matrix element ? 

• If the magnitude of the expected value is much greater than 
the square root of the variance of that element then the 
previous method will work. 

• If the expected value is approximately zero, then the off- 
diagonal element will have, approximately, a Gaussian 
distribution. 

• If neither of the above is the case then the associated 
correlation coefficient of the element in question should be 
investigated. There is already an approximate, known 
distribution associated with the sample correlation coefficient. 
(This may also work for the two previous conditions as well.) 
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Empirical Error Covariance Matrix 


Other Comments 

All of the confidence interval discussion revolves around theoretical 
expectations. Thus the confidence intervals formally describe the 
theoretical uncertainty in the theoretical state error covariance 
matrix. 

It is possible to determine "empirical" confidence intervals and this 
should result in "K-factor" like modifications to the variances of the 
individual covariance elements. 

Future work and applications... 
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Summary 


• It is possible to directly and unambiguously determine a formally 
correct empirical state error covariance matrix which describes the 
batch filter estimate state vector uncertainty. 

• It is directly possible to determine theoretical confidence intervals 
associated with the elements of the batch state error covariance 
matrix. These intervals apply specifically to the traditional batch 
filter error covariance matrix and, though not presented, it is 
possible to determine confidence intervals specifically for the 
empirical error covariance matrix elements. 

• All of this within one estimate without any knowledge of the true 
state nor having to perform any systematic comparisons. 
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